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Abstract 

Computing moments of various parameter estimators related to an au¬ 
toregressive model of Statistics, one needs to evaluate several expressions 
of the type mentioned in the title of this article. We proceed to derive the 
corresponding formulas. 


1 Introduction 

The autoregressive model of Statistics generates a random sequence of observa¬ 
tions by 

Xi = OiiXi-i + a2Xi-2 + ■ • • + OikXi—k + Si ( 1 ) 

where Si are independent, Normally distributed random variables with the mean 
of 0 and the same standard deviation, and A: is a fixed integer, usually quite small 
(e.g. k = 1 defines the so called Markov model). The sufficient and necessary 
condition for the resulting sequence to be asymptotically stationary is that all 
k solutions of the characteristic polynomial 

+ a2X’^-^ + ■ ■ ■ + ak ( 2 ) 

are, in absolute value, smaller than 1 (this is then assumed from now on). 

The jth-order serial correlation coefficient pj (between Xi and Xi^j) is then 
computed by 

Pj = + A2A2^^ + • • • -I- AfcAjf ^ (3) 

where the Ai’s are the k roots of and the Ai coefficients are themselves 
simple functions of these roots. 


1 


Computing the first few moments of various estimators (of the ai parame¬ 
ters) boils down to computing moments of expressions of the 

n 

(4) 

and 

n-j 

(5) 

i=l 

type, where Xi, X 2 , ■ ■ ■ Xn is a collection of n consecutive observations (assum¬ 
ing that the process has already reached its stationary phase). 

This in turn requires evaluating various summations (see m), of which the 
most difficult are 


n 

\ 1 * 1 —i2-l-si| I |i2-il-l-S2| 
Ai A 2 

il,i2 = l 



E 


\ 1 * 1 —*2-|-Sl| > 1*2—*3-|-S2| \ |*3-*1-|-S3| 

Ai A2 Ag 


*1,*2,*3 = 1 


( 6 ) 

(7) 


and 


E 


\ |il-i2 + Sij A ji2-^3 + S2| xKs-U + SsI \|i4-il+S4| 
Ai A2 A3 A4 


n 5^2 1^3 1^4 — 1 


( 8 ) 


where Ai, A 2 , A 3 , and A 4 are any of the Ai roots (some may appear in duplicate), 
si) ■52, .S 3 , and S 4 are small integers, and h indicates that each of the upper limits 
equals to n, perhaps adjusted in the manner of ([5]). 

It is possible (but rather messy — the result depends on the values of si, S 2 
and n — see m) to exactly evaluate m and realize that the answer will always 
(this goes for the other two summations as well) consist of three parts: 


1. terms proportional to A", which tend to zero (as n increases) ‘exponen¬ 
tially’, 

2. terms which stay constant as n increases, 

3. terms proportional to n. 


Luckily, to build the approximation which is usually deemed sufficient (see 
DP), we need to find only the n proportional terms. These can be extracted 
by dividing the relevant summation by n and taking the n —>■ 00 limit. Inci¬ 
dentally, this results in the following (and most welcomed) simplification: the 
corresponding answer will be the same regardless of the n adjustments (thus, 
we may as well use n instead), and will similarly not depend on the individual 
Si’s, but only on the absolute value of their sum. The proof of this statement is 
omitted. 
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2 Evaluating the limits 

Starting with ([6]), we obtain 


F2(M,A2;S) 


= lim - y A 

ra—>oo n 


|il-i2+sil \ K2-il+S2| 
1 ^2 


il,i2 = l 


= lim - y 

n—>^oo 77, 


Kl—^2 I ^^2—■^l+'S'l 


n min(n,n-j) 

1 xliKI5-j| 


= lim - y y 

rj.—^on T) • ^ ^ 


A-A^ 


= lim y 

rt .—^oo t ^ 


il,i2 = l 

~ b1 \bl \l‘S'-t| 


n—¥oc Tl 

j^-n 


j= — n zi=max(l,l—j) 
oo 

^2 “ 2-^ '^1 ^2 
j^-oo 

S oo 


= E v aE+ E^i^2“^+ E AiA^, 

j=l j=S+l 


-s 


j=-aD 


Af+i(l-A2) 


Ay(l-A?) 


(Ai — A2)(1 — A 1 A 2 ) (A 2 — Ai)(l — A 2 A 1 ) 


where S = |si + S 2 |- Following the usual convention, an empty summation (such 
as Ej=i) ^ value. 

Note that the answer can be written in the following form: 


Ea: 

2=1 


S+^-1 


n 


1-A| 


i=i 


(Ai ^j)(l ^i^j) 


(9) 


with £ = 2. Also note that, when A 2 = Ai, the value of F 2 (Ai,Ai;S') can be 
easily obtained by 


lim F2 (Ai,A2;5') 

A2^Ai 


Af (l + ^ + (l-^)A2) 

r^Af 
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2.1 The case of 3 A’s 

Moving on to o, we now get 


i^3(Ai,A2,A3;5)= lim - V 

rj .—n • ^ 


|il-j2+Sl| -v 1*2—i3+S2| -i |i3-il+S3l 

^2 ^3 


*1,*2,*3 = 1 


= lim i y A^-*^Ia'*=-*^Ia 

n—^oo Ji ^^ 


Ki-* 2| ■^|*2-*3| \l*3-il+S| 


*1:*2,*3 = 1 


1 


min(n,n—ji) min(n,n—J 2 ,n—ji — J 2 ) 


ji= — 7 i j2=m.ax( —n,—n—ji) n=m.ax(l,l—j2,l—ji —j2) 
n min(n,n-ji) 

= lim y y 

n —^00 ^ ^ 

tl=—n j 2 =max(—n, —n—ji) 

00 00 

Jill \li2l \IS-j1-j2l 


min(n,n-j 2 ,n-ji-j 2 )-ma.x(l,l-j 2 ,l-ji-j 2 } + l \ Ji I \li 2 l \IS-j 1 -j 2 l 
n ■ -^1 ^2 -^3 


= E E 'A^ 


J 1—-00 J 2——00 


where S' = |si + S2 + S3|- 

This time, evaluating the last summation is slightly more difficult; we will 
do it quadrant by quadrant. 

For the first quadrant {including the adjacent half-axes and the origin), we 
get (visualize the quadrant, cut by the S = ji + j2 line): 


S s-ji S 00 

E E Af Ar + E E Af A? 

ii=oi 2 =o ji=o j 2 =s-ji+i 

Af+ 2 ( 1 -Ai) 


00 00 


E E-'tSv, 

ii=s-i-i j 2 =o 


l+j2—S 


Ay(i-Ai) 


A 


S-l-2 


(Ai — A2 )(Ai — A3)(1 — A1A3) (A2 — Ai)(A2 — A3)(1 — A2A3) (A3 — Ai)(A3 — A2) 


For the second quadrant (again, including the corresponding boundaries - the 
resulting duplication with the first quadrant will be removed later), the same 
kind of approach yields 


0 S—ji 0 00 

E E AyA^/Af-^-^-^-^+ y y AyA^/A^ 3 ^+^=-^ 

il = -00 ^2=0 il = -00 j2=S-il-|-l 

_ A|'+^( 1 -A§) Af+^ 

(A2 — A3)(1 — A2Ai)(1 — A2A3) (A3 — A2)(1 — A3A1) 


The fourth quadrant clearly results in the same answer, with Ai and A2 inter¬ 
changed, namely 

Af+^( 1 -A^) Af+^ 

(Ai — A 3)(1 — AiA 2)(1 — A1A3) (A3 — Ai)(l — A3A2) 
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Finally, the third quadrant (including its boundaries) contributes 


0 


0 

E 

jl— — CC j 2 — -00 




Ai 




\s-j1-j2 


Af 


(1 — A3Ai)(1 — A3A2) 


Adding the four results does not yield the desired answer, since the contri¬ 
bution of each of the two axes has been included twice, and that of the origin 
altogether four times. This can be easily corrected by subtracting f2(A2, A3; S) 
which removes the extra contribution of the ji = 0 axis, and T2(Ai,A3;S') 
which does the same thing with the j2 = 0 terms. This leaves us with the origin 
(ji = j2 = 0) which, at this point, is still contributing double its value (two 
contributions have been removed with the two axes); subtracting Af fixes that 
as well. 

The final answer thus becomes 


T3(Ai, A2, A3; S) — 


Af+^( 1 -Ai)(l-Ai) 

(Ai — A2 )(Ai — A3)(1 — AiA2)(1 — 
_ A|+"( 1 -Af)(l-Ai) 

(A2 — Ai)(A2 — A3)(1 — A2Ai)(1 
Af+^( 1 -Af)(l-Ai) 
(A3 — Ai)(A3 — A2)(1 — A3Ai)(1 


A1A3) 


— A2A3) 


— A3A2) 


Note that this has the form of ([ 9 |) with £ = 3 . 

When any of the three A‘s are identical, the answer can be found as the 
corresponding limit of the previous expression. Thus, for example 


F 3 {X,X,X-,S) 


2 + 3 S + S^ + 2 { 4 -S'^)X^ + { 2 - 3 S + S^)X'^ 

2(1-A2)2 ^ 


etc. 

2.2 The case of 4 A’s 

The main challenge is to evaluate the last limit, namely 


F4(Ai, A2, A3, A4; S) 


= lim — 

n—>-oo n 


E 


■vKl— i 2 +Sl| \|i2-j3 + S 2 | ■! hs-U-l-sal -I |i4-n-l-S4| 
Ai A2 A3 A4 


n A 2 1*3 1*4 —1 






\lii| \b'2| X b'3| \ l‘S-ii-t2-t3l 
Ai A 2 Aci A 3 


41 = -00 42 = -00 43=-00 

We proceed octant by octant; the octants will be identified by the signs of the 
ji, j2 and js indices, respectively. 
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For the first octant denoted 0 +++ (including the adjacent portions of coor¬ 
dinate planes, axes and the origin), we get 

s s-ji s-ji-n 

E E E Af 

S S—j\ oo 

+ E E E Af A^/Ai^Ai^+^=+^-^ 

jl=0 h=0 j3 = S-jl-j2-|-l 
S oo oo 

+ E E E Af A^A? 

n=0i2=S-ii-i-i j3=o 


OO OO OO 


3 \jl-\-j2-\-j3 — S 

4 


+ E EEd‘A?A"Ai 

ii=s-i-i j 2 =o j 3 =o 

_Af+"( 1 -Al)_ 

(Ai — A2 )(Ai — A3)(Ai — A4)(1 — A1A4) 

, _ Af+^(1 - AI) _ 

(A2 — Ai)(A2 — A3)(A2 — A4)(1 — A2A4) 

, _ Af+^( 1 -Al) _ 

(A3 — A2)(A3 — A2)(A3 — A4)(1 — A3A4) 

^^4 


Af +3 


(A4 — Ai)(A4 — A2)(A4 — A3) 

To understand why it was necessary to break the summation into four parts, it 
helps to visualize the first octant, cut by the S = ji + j2 + js plane,thus: 
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Our brain can interpret this image in two different ways; please make an effort 
to see the triangle as the most distant part of the picture. 

As the next octant we take 0 +-|_(with all its boundaries), contributing 


S s-ji 0 

E E E Af 

jl^O j2^0 j3^-00 

OO OO S—jl—j2 

+ E E E 

jl=0 j2 = S-jl + l j3=-OC 
OO OO 0 

+ E E E AfArA3-^-^Af+^-^+^-^ 

jl^o j2^S-ji-\-l j3^S-ji-j2 + l 

OO OO S—jl—j2 

+ E E E AfArA3-^^Af-^-^-^-^ 

jl=S+lj2=0 j3 = -OC 


OO OO 0 

\il \i2 \-j3 \jl+j2+j3-S 
^1 ^2 ^3 \ 

jl=s+lj2=0 j 3 = S-jl -j2 +1 



_ Af+^( 1 -Al) _ 

(Ai — A 2 )(Ai — A 4)(1 — AiA 3)(1 — A 1 A 4 ) 

_ Af+^( 1 -Al) _ 

(A 2 — Ai)(A 2 — A 4)(1 — A 2 A 3)(1 — A 2 A 4 ) 


(A4 — Ai)(A4 — A2)(1 — A4A3) 

Again, visualizing the situation may help (the corner being the most distant 
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part of the picture): 



The O _and O.)_^ octants contribute the same expression each, after the 

As ■(-)• Ai and As •<->• A2 interchange, respectively. 

For O _I- (including boundaries) we get 

0 0 

E E E 

jl = -00 j2 = -00 j3=0 

0 0 CO 

+ E E E Ar^^A2^^ArAi^+^^+^'^-^ 

il=-00 t2 = -00 j3=S-jl-j2 + l 


xl+\i-xl) 


(As — A 4)(1 — AsAi)(l — AsA 2)(1 — ASA 4 ) 


+ 


A 


s+i 


(A 4 — As)(l — A4 Ai)(1 — A 4 A 2 ) 
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because this is how it looks like (again, the corner to be seen as most distant) 



and similarly for _and 0 _h _, after the A3 ■(-)• Ai and A3 A2 interchange, 

respectively. 

Finally, O _with its boundaries contributes 

E E E 

31 —-00 j2 — -00 j^—-00 

^ _A|_ 

(1 — A4Ai)(1 — A4A2)(1 — A4A3) 

Adding the eight results and subtracting F3(A2, A3, A4; S')+F3(Ai, A3, A4; 5 ') + 
F3(Ai, A2, A4; S') to remove the duplicate contribution of the three coordinate 
planes; further subtracting F2(Ai, A4; S) + F'2(A2, A4; S) + F2(A3, A4; S) to remove 
the originally quadruple (now duplicate) contribution of the three axes; and 
finally subtracting Af to remove the remaining, originally eightfold (now dupli¬ 
cate) contribution of the origin, yields the final formula for ^^(Ai, A2, A3, A4; S). 
Not surprisingly, it turns out to be equal to ([S]) with £ = 4 . 

2.3 Further challenge 

At this point, it is fairly obvious that F5(Ai, A2, A3, A4, A5; S) will be given by 
with I = b, etc. To prove this by the technique of this article becomes 
increasingly more difficult (impossible in general, since € can have any integer 
value). One clearly needs to proceed by induction - would anyone want to try? 
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